Issue 387 resolved #395
Open
Issue 387 resolved #395
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
To solve the problem of the head dimension exceeding the shared memory limit, we need to add a check after the line where
d_inneris calculated. Ifd_innerexceeds a safe maximum value, we should set it to that maximum value.To solve the problem of the head dimension exceeding the hardware limits, we need to add a check in the
__init__methods of bothMixerModelandMambaLMHeadModelclasses. This check will ensure that the head dimension (d_model) does not exceed a certain limit. If it does, it will adjust it to the maximum allowable value based on the hardware.To solve the problem, we need to add a parameter to configure the head dimension (headdim) and ensure it is set appropriately. We also need to validate the head dimension to ensure it does not exceed hardware limits. Additionally, we need to adjust memory allocation and kernel function calls to use the configured head dimension and ensure memory usage is optimized.